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Abstract 

Blackwell games are infinite games of imperfect information. The two 
players simultaneously make their moves and are then informed of each 
other's moves. Payoff is determined by a Borel measurable function / 
on the set of possible resulting sequences of moves. A standard result in 
Game Theory is that finite games of this type are determined. Blackwell 
proved that infinite games are determined, but only for the cases where 
the payoff function is the indicator function of an open or Gs set |^ ^ . For 
games of perfect information, determinacy has been proven for games of 
arbitrary Borel complexity ^, |^ ^ • In this paper I prove the determinacy 
of Blackwell games over a Gsa set, in a manner similar to Davis' proof of 
determinacy of games of Gscr complexity of perfect information j^. 

There is also extensive literature about the consequences of assuming 
AD, the axiom that all such games of perfect information are determined 
1^, 0. In the final section of this paper I formulate an analogous axiom 
for games of imperfect information, and explore some of the consequences 
of this axiom. 

1 Introduction 

Imagine two players playing a game of Blind Chess. The only board they have is 
in their minds, and they make their moves merely by announcing them. Someone 
who doesn't know the rules would find a game like this difficult to follow. If 
that someone was of a literal bend, he might describe it like this: 

"There were two players, playing against each other. The first player 
said something, and I was told it was her move, and that she had 
made the move by saying it. The other player thought for a while, 
and then announced his own move. Then the first player made a 
move again, then the second player, and so forth. The moves always 
sounded similar, something like "pawn from ee-four to ee-five". So I 
think they couldn't just say anything, but had to select their moves 



*My thanks go to Michiel van Lambalgon and Tonny Hurkens, for their guidance, ideas and 
meticulous proofreading. This research was partially funded by the Netherlands Organization 
for Scientific Research (NWO), under grant PCS 22-262. 
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from only a few possible options. And suddenly they stopped, and 
shook hands, and I was told that the first player had won, apparently 
because of the moves she and her opponent had played." 

If no one gave the poor fellow a copy of the rules of Chess, the way a sequence 
of moves determines which player wins would probably seem quite arbitrary. 
And our hypothetical observer might be quite impressed that apparently chess- 
players are able to memorize this long list of what the result is of each possible 
sequence of moves. 

Of course, the game of Chess is not really that arbitrary, and those of us 
who play chess only need to know a few simple rules to figure out which player 
has won. But we can use this concept of a game to construct a quite general 
mathematical game Tp,i,(f). 

Let there be given two finite sets X and Y, an integer n, and a 
function / assigning to each sequence w of length n of pairs {xi ,yi) €: 
X X Y, a. payoff f{w) G M. Two players are playing against each 
other. Each player, in turn, makes a move by selecting an element 
Xi € X or yi € Y, respectively, and announcing his or her selection. 
Then they each in turn make a second move, and a third move, 
and continue making moves until n rounds have been played. This 
generates a sequence w of length n of pairs {xi, yi) G X x Y. Then 
they stop, and player II pays player I the amount f{w). 

With the right choices for X, Y, n and /, the game Tp^iXf) can 'emulate' the 
game of Chess. For if we let X and Y be the set of all possible chess moves, and 
n — 6350|^, then a sequence w corresponds to a finished game of chess. We now 
set /{w) = 1, f{w) — 0, or /(w) — \, depending on whether the corresponding 
game is a win for White, a win for Black, or a draw.^ And voila, we have our 
Chess emulator. 

But Chess is not the only game that can be 'emulated' in this manner. The 
same can be done with Noughts-and-Crosses, Connect-Four, Go and Checkers. 
In general, the games Fp. ,;.(/) can emulate any game G that has the following 
properties: 

• There are two players. 

• There is no element of chance 

• Moves are essentially made by selecting them and announcing them. 

• There is no hidden information: a player knows all the moves made so far 
when making her current move, and there is nothing going on simultane- 
ously either [Perfect Information) . 

^The fifty-move rule is a rule in chess stating that if no piece has been captured and no 
pawn has been moved for fifty turns, the game is a draw. Under the fifty-move rule, a game 
of chess can last a maximum of 6350 moves. 

^ If ui does not correspond to a legal chess game, we count it as a win for White if the first 
illegal move is made by Black, and vice versa. 
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• If one player loses (a certain amount) the other player wins (that same 
amount) [Zero- Sum). 

• The game can last no more than a certain number of rounds (Finite Du- 
ration). 

• There is a maximum number of alternatives each player can select from 
(Finite Choice- of- Moves). 

Any results for the games Fp.i. (/) apply to all the games with these properties. 
David Blackwell described the concept of a strategy as Q : 

Imagine that you are to play the White pieces in a single game of 
chess, and that you discover you are unable to be present for the 
occasion. There is available a deputy, who will represent you on the 
occasion, and who will carry out your instructions exactly, but who 
is absolutely unable to make any decisions of his own volition. Thus, 
in order to guarantee that your deputy will be able to conduct the 
White pieces throughout the game, your instructions to him must 
envisage every possible circumstance in which he may be required to 
move, and must specify, for each such circumstance, what his choice 
is to be. Any such complete set of instructions constitutes what we 
shall call a strategy. 

Thus, a strategy for a given player in a given game consists of a specification, for 
each position in which he or she is required to make a move, of the particular 
move to make in that position. In turn, a position can be specified by the 
moves made to get to that position. If we apply this to the game ^p.i.if), a 
strategy becomes a function from the set of sequences of length < n of pairs 
{xi, yi) £ X X Y, to the set of possible selections X, Y respectively. 

Given strategies for each of the players, the outcome of the game is deter- 
mined: each move follows from the current position and the strategy of the 
player whose turn it is to move, and determines the next position. So, the to- 
tality of all the decisions to be made can be described by a single decision - 
the choice of a strategy. This is the normal form of a game: the two players 
independently make a single move, which consists of selecting a strategy, and 
then payoff is calculated and made. 

Of course, there are good strategies and bad strategies. The value of a 
strategy for a given player is the result of that strategy against the best coun- 
terstrategy. The value of a game for a given player is the best result that that 
player can guarantee, i.e. the value of that player's best strategy. A game is 
called determined if its value is the same for both players. That value is the 
result that will occur if both players are playing perfectly.^ 

•^In more general cases, we allow e-approximation, i.e. a game is determined iff there exists 
a value v such that for any e > 0, the two players have strategies guaranteeing them a payoff 
of at least u — e or at most v -\- e, respectively. 
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Victor Allis [y recently demonstrated that in a game of Connect-Four, the 
first player can win, i.e. has a strategy that wins against any counterstrategy. 
And countless persons throughout the ages have independently discovered that 
in the game of Noughts-and-Crosses, both players can force a draw. These are 
both examples of determinacy. It can be shown (using induction) that any game 
r(/) as defined above is determined, and hence any game with all of the prop- 
erties mentioned above is determined. In the case of Go, Chess, and Checkers, 
this means that either one of the players has a winning strategy, or both players 
have a drawing strategy. 

Now consider the game of Scissors-Paper-Stone. In this game, the two play- 
ers simultaneously 'throw' one of three symbols: 'Stone' (hand balled in a fist), 
'Paper' (hand flat with the palm down) or 'Scissors' (middle and forefinger 
spread, pointing forwards). If both players throw the same symbols, the result 
is a draw; otherwise. Paper beats Stone, Stone beats Scissors, and Scissors beats 
Paper (the reason being that "Paper wraps Stone, Stone blunts Scissors, and 
Scissors cut Paper"). In this game, the players do not make moves in turn, but 
simultaneously. In other words, both players make moves, and neither player 
knows what move the other is making. This is an example of a game of Imperfect 
Information. 

The strategy 'Throw Stone' is a losing strategy, because it loses against 
the counterstrategy 'throw Paper'. The same can be said for any strategy of 
the type 'throw this', for both players. So in terms of the concept of strategy 
described above, this game is not determined. On the other hand, consider the 
'strategy' 'throw Scissors 1/3 of the time, throw Paper 1/3 of the time, and 
throw Stone the remaining 1/3 of the time'. Against any other strategy, this 
strategy loses, draws and wins 1/3 of the time each, for an 'average result' of a 
draw. This strategy does not fit in the concept of strategy given above, but it 
is clearly worth considering. 

Strategies of this new type are called mixed strategies, as opposed to the old 
type of strategies, the pure strategies. A mixed strategy for a given player in 
a given game consists of a specification, for each position in which he or she is 
required to make a move, of the probability distribution to be used to determine 
what move to make in that position.^ Given mixed strategies for each of the 
players, the outcome of the game is not determined, but we can calculate the 
probability of each outcome. If we assign values to winning and losing ('the loser 
pays the winner one dollar'), then we can calculate the average profit/loss one 
player can expect to make from the other, playing those strategies. 

The value of a mixed strategy is therefore the expected average result against 
the best counterstrategy. And a game is called determined if, for some value 
V, one of the players has a strategy with which she can always expect to make 
(on average) at least v, no matter what the other plays, while the other player 
has a strategy with which he can always expect to lose (on average) at most v, 



^ Standard game theory defines a mixed strategy as a probability distribution on pure 
strategies, but the above definition can be shown to be equivalent to that one. 
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no matter what the other plays. As before, it can be shown (using induction 
and a theorem of Von Neumann) that aU finite two-person zero-sum games with 
Imperfect Information (i.e. the games with the properties mentioned above, 
except that players make moves at the same time instead of one after the other) 
are determined. 

All the games mentioned so far are of finite duration. Let, as before, X and 
Y be two finite sets, and let / be a function assigning to each countably infinite 
sequence w of pairs {xi,yi) £ X x Y, a payoff f{w) G We first consider 
games of infinite duration and perfect information: 

Two players are playing against each other. Each player, in turn, 
makes a move by selecting an element xi E X oiyi G Y, respectively, 
and announcing his or her selection. Then they each in turn make 
a second move, and a third move, and continue making moves for 
a countably infinite number of rounds. This generates an infinite 
sequence w of pairs {xi,yi) E X xY. 'Then' they stop, and player 
II pays player I the amount f{w). 

The problem with infinite games, of course, is that the outcome is only known 
after an infinite number of moves, and thus it is impractical to play the game as 
it is. But our concept of a strategy as a specification of which move to make in 
each position, is still valid in the case of games of infinite duration. And given 
strategies for both players we can construct the infinite sequence of moves that 
will be played (or the probability distribution thereof), and apply the payoff 
function to obtain our (expectation of the) outcome. Hence we can still play 
the game in a fashion, by using its normal form. 

The concepts of values and determinateness carry over as well. But it is no 
longer provable that all such games are determined. For some payoff functions 
/, such as bounded Borcl- measurable functions /, it has been proven that the 
infinite game of perfect information Fp.i. (/) is determined. But using the Axiom 
of Choice, a nonmeasurable payoff function / can be constructed such that r(/) 
is not determined |]l0). The axiom AD, the axiom that all games r(/) are 
determined, is widely used as an alternative to AC H, [Tl|| . 

A game of infinite duration and imperfect information is similar, except that 
both players make their n*'* move at the same time. These games are called 
Blackwell games, named after David Blackwell, the first one to describe and 
study these games For Blackwell games, it has been proven that r(/) is 
determined for the case that / is the indicator function of an open or Gs set. 
In this article I prove determinacy of r(/) for the case that / is the indicator 
function of a Gsa set. But the general case of Borel-measurable functions is still 
open. 



^ We tacitly assume / to be bounded, as otherwise things get ugly. 
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2 Definitions, Lemmas and Terminology 
2.1 Games, Strategies and Values 

The definitions in this subsection are fairly standard, and merely formalize the 

intuitive concepts from the introduction. The lemmas arc all basic properties 
of game- values. For reasons of conciseness, no proofs are given in this section. 

In order to define what a Blackwell game is, we first need some sets. Let X and 
Y be two finite, nonempty sets, and put Z = X x Y. An (infinite) play is a 
countably infinite sequence w of pairs {x, y) G Z. We write W for the set of all 
plays, i.e. W = Z^ 

Definition 2.1 Let / : — > JR be a bounded Borel (measurable) function (i.e. 
a bounded function such that is a Borel set for every open set O C M). 

The Blackwell game T{f) with payoff function f is the two-person zero-sum 
infinite game of imperfect information played as follows: Player I selects an 
element xi G X (makes the move x\) and, simultaneously, player II selects an 
element yi G Y. Then both players are told zi = {xi,yi), and the game is at 
or has reached position (2:1). Then player I selects X2 & X and, simultaneously, 
player II selects 2/2 G Y- Then both players are told Z2 = {x2, 2/2), and the game 
is at position (21,^2)- Then both players simultaneously selects x^ E X and 
t/3 G Y, etc. Thus they produce a play w = {zi, Z2, ■ ■ ■)■ Then player II pays 
player I the amount f{w), ending the game. 

A position or finite play {of length k) is a finite sequence p (of length k) of pairs 
{x,y) G Z. We write P for the set of all positions, i.e. P = Z^'^ . 

Some notation and terminology that we are going to use: 

w usually denotes an infinite play, p denotes a finite play or position. 

W|„ denote the sequences consisting of the first n moves made in p, w re- 
spectively (counting a pair {x,y) as one move). 

P^p' ^ P^w denote the sequences consisting of the finite sequence p followed by 

the finite sequence p' or the infinite sequence w, respectively. 

len(p) denotes the length of a finite sequence p. 

e denotes the position of length 0, i.e. the empty sequence. 

Wn denotes the set of all finite plays of length n, i.e. Wn = Z^, for n G iV. 

p Cw denotes that 'U'iien(p) = P) and we say that w hits or passes through p. 

p C p' denotes that pjien(p) — P P' 7^ P' '^^ P' follows p, and p 

precedes p'. 

p '!=p' denotes that p'icn(p) ~ P^ ^^'^ that p' follows or is equal to p. 

\p] denotes the set {w e W \ w D p} of all plays hitting the position p. 
[H] denotes the set {w G W \ 3p € H : w D p} of all plays hitting any position 
in a set of positions H. 

We will sometimes write a sequence ((xi, yi), (x2, 2/2), . . .) as {xi,yi,X2, 2/2, • • •)• 
T{S) denotes the game T{Is), where Is is the indicator function of 5 C W. 
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Remark 2.2 Wc give W the usual topology by letting the basic open sets 
be the sets of the form [H] for some some set H C W„ of positions of fixed 
length n. Then the open subsets of W are exactly those of the form [H] for 
some set H of positions. The Gs subsets of W arc exactly those of the form 
{■u) G I #{p G H \ w hits p} = 00} for some set H of positions. Note that 
under this topology, is a compact space. 

Definition 2.3 A strategy for player I in a Blackwell game r(/) is a function 
a assigning to each position p a probability distribution on X. More formally, 
(T is a function P [0, l]'^ satisfying \/p <E P : '^j.^x '^ip)i-'') = 1- 

Analogously, a strategy for player II is a function r assigning to each position 
p a probability distribution on Y. 

Definition 2.4 Let a and r be strategies for players I, II in a Blackwell game 
r(/). cr and T determine a probability measure /U^.r on W, induced by 

n 

M<^,rb] = P{'^ I hits P} = n • T{P\ii-l))iyi)) (2-1) 

for any position p= (xi, yi, . . . , a;„, ?/„) e P. 

The expected income of player I in r(/), if she plays according to a and 
player II plays according to r, is the expectation of f{w) under this probability 
measure: 

E{a vs T in r(/)) = J f{w)dfx,,r{w) (2.2) 

Definition 2.5 Let r(/) be a Blackwell game. The value of a strategy a for 
player I in r(/) is the expected income player I can guarantee if she plays 
according to a. Similarly, the value of a strategy r for player II in r(/) is the 
amount to which player II can restrict player I's income if he plays according to 
T. I.e. 

val(CTinr(/)) = infi;((7vsr inr(/)) (2.3) 

r 

val(rinr(/)) = sup£((T vs r in r(/)) (2.4) 

cr 

Definition 2.6 Let r(/) be a Blackwell game. The lower value of r(/) is the 
smallest upper bound on the income that player I can guarantee. Similarly, the 
upper value of r(/) is the largest lower bound on the restrictions player II can 
put on player I's income. I.e. 

val^(r(/)) = supval(cr in r(/)) = supinf i;((T vs r in r(/)) (2.5) 
val^ (r(/)) = inf val(r in r(/)) = inf sup E{a vs r in r(/)) (2.6) 

Clearly, for all games r(/), val^(r(/)) < vaf (r(/)). If valT(r(/)) = val^(r(/)), 
then r(/) is called determined, and we write val(r(/)) = val^(r(/)) = val'''(r(/)). 
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Definition 2.7 Let r(/) be a Blackwell game, and let e > 0. A strategy a 
for player I in r(/) is optimal if val{a in r(/)) = val'''(r(/)). A strategy a for 
player I in r(/) is e-optimal if val(cr in r(/)) > val^(r(/)) — e. Similarly, a 
strategy t for player II in r(/) is optimal if val(r in T{f)) = val^(r(/)), and 
e-optimal if val(r in r(/)) < ybI^ {T{f)) + e. 

Some basic properties of these values are: 

Lemma 2.8 Let f , g be two payoff functions such, ttiat for all w € W , /{w) < 
g{w). Then v8l^{T{f)) < val^(r(c/)) and val^(r(/)) < va.\\r{g)). 

Lemma 2.9 Let f be a payoff function, and let a,c G lR,a > O.Then 
val^(r(o/ + c)) = o val^(r(/)) + c and val^(r(a/ + c)) = o valT(r(/)) + c. 

Lemma 2.10 Let f be a payoff function, and let fsw ■ (Y x X)^ M be 
defined by 

fsw{{yi,xi), {y2,X2), ...) = f{{xi,yi), {X2,y2), ■ ■ •) (2.7) 

Then 

Yal^{r{-f)) = -yal^TsMsn,)) (2.8) 

val1'(r(-/)) = -val^(r,„(/,^)) (2.9) 

where Tgwifsw) is the Blackwell game with payoff function fsw in which player 
I selects moves from Y and player II selects moves from X. 



Lemma 2.11 Let {fi)j he a sequence of functions fi '-W ^ [a, 6] such that {fi)i 
converges pointwise to a function f : W ^ [a,b]. Then for any two strategies 
a, T, lim,^oc E{g vs t in r(/j)) = E{a vs r in r(/)) 



2.2 StEirting and Stopping 

Definition 2.12 Let / : ^ J? be a boimdcd Borcl function, and p = 
{{xi,yi), {x2,y2), ■ ■ • , {xnjVn)) & positiou. The subgame T{f,p) starting from 
position p is the game played like r(/), except that the players start at round 
n + 1, and the first n moves are supposed to have been .xi, yi, X2,y2, ■ ■ ■ , Xn, yn- 
The game r(/,p) is played exactly the same as the game T{g), where g is the 
payoff function defined by ^(w) = f{p^w). 

As before, strategies a and r determine a probability measure /i^^^ r(/.p) 
on W . This measure is equal to the conditional probability measure obtained 
from fi^^T given [p], i.e. 

Ma,r inr(/,rt(g) = ^"'^^^!^i^^^ ifM.,rb]=0 (2.10) 

The expected income of player I, the value of a strategy cr, etc. are defined for 
the games T{f,p) in the same manner as for the games r(/). 
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Definition 2.13 A stopping position in a Blackwell game r(/) is a position p, 
such that for all plays w,w' G [p], f{w) — f{w'). We will denote this value by 
f{p)- A stopset in a Blackwell game r(/) is a set H of stopping positions, such 
that no stopping position p ^ H precedes another stopping position p' G H . 

We will often define a payoff function / using the following format: 

f(p) = formulal for p £ H 
/(w) — formula2 ii w ^ [H] 

where iJ is a set of positions such that no position p E H precedes another 
position p' G H. Then H is a stopset in the game Tnif)- 

Remark 2.14 If p is a stopping position, any moves made at or after p will 
not affect the outcome of the game. It is often convenient to assume that both 
players will stop playing if a stopping position is reached. If r(/) is a Blackwell 
game, and iJ is a stopset, we write Tnif) to exphcitly denote that players stop 
playing at the positions in H. In this case, we only require strategies to be 
defined on nonstopping positions. Similarly, with respect to a subgame T{f,p), 
we only require strategies to be defined on positions that are following or equal 
to p. In fact it is occasionally necessary to assume that a strategy is not defined 
on positions outside the subgame proper, for instance to combine strategies for 
different subgames into one big strategy. 

Using stopsets, a finite game can be treated as a special type of infinite game. 
For finite games, we have determinacy, as well as a kind of continuity of the 
value function. 

Definition 2.15 Let r(/) be a Blackwell game. If, for some n, all positions in 
Wn are stopping positions, then r(/) is called finite (of length n). If r(/) is 
finite, we can stop after playing n rounds, and we will denote this by writing 

rn(/). 



Theorem 2.16 (Von Neumann's Minimeix Theorem|12|) LetTi{f) he a 
finite one-round Blackwell game (i.e. of length 1). Then Ti{f) is determined, 
and both players have optimal strategies. 



Theorem 2.17 Let Tn{f) be a finite Blackwell game of length n. Then r„(/) 
is determined, and both players have optimal strategies. 



Lemma 2.18 Let n G IN . Let {fi)i be a sequence of payoff functions ft : 
Wn [a, b] such that {fi)i converges to a payoff function f : Wn [a, b]. Then 
val(r„(/)) = lim.^oo val(r„(/,)). 
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2.3 Equivalent Truncated Subgames 

In games like Chess, Go, or even Risk or Monopoly, a player is usually allowed 
to give up if he has no hope of winning. He doesn't have to play it out in 
the hope that the other player will make a mistake. Two players can agree 
beforehand to stop in certain positions, and pay out the value of the game at 
that position rather than continue playing. Provided their assessment of that 
value is accurate, this does not change the value of the total game. We will call 
a game resulting from such an alteration a truncated subgame. 

Definition 2.19 Let /, g be two payoff fmictions, and H a stopset in T{g). 
Tnig) is an equivalent truncated subgame of r(/) {truncated at H), if for any 
play w ^ [H], f{w) = g{w), and for any p € H, g{p) = val(r(/,p)). TH{g) is 
a truncated subgame, equivalent for player I [player II], if for any play w ^ H, 
f{w) = g{w), and for any pGH, g{p) = val^(r(/,p)) [g{p) = valT(r(/,p))]. In 
all three cases, r(/) is called an extension of TH{g)- 

Note that TH{g) is an equivalent truncated subgame of r(/) iff it is a truncated 
subgame equivalent for both player I and player II. 

Lemma 2.20 Let T{f) be a Blackwell game, and let Tnig) be a truncated sub- 
game of r(/), truncated at a set of positions H, equivalent for player I [for 
player II[. Then val^ (T (f)) ^ val^iT nig)) [val\T{f)) = va\^ (Tnig))]. FuHher- 
more, for any e > 0, any e-optimal strategy for player I [player II] in (5) (if 
it is undefined on all positions at or after positions in H) can be extended to an 
e-optimal strategy for player I [player II[ in r(/). 

Sketch of proof: We find an e-optimal strategy for the truncated subgame 

Tnig), and for the appropriate S, (^-optimal strategies for the games r(/,p) 
starting at positions p £ H, i.e. the positions where Li/ (g) stops. Then we 
tie them together, and calculate how well the combination strategy performs 
against opposing strategies. 

Corollary 2.21 Let T{f) be a Blackwell game, and let Tnig) be an equivalent 
truncated subgame ofT{f) (truncated at H). If T nig) is determined, thenT{f) 
is determined, and val(r(/)) = val(r// ((;)). Furthermore, for any e > 0, any 
e-optimal strategy for player I or player II in Tnig) can be extended to an e- 
optimal strategy for player I or player II in r(/). 

Corollary 2.22 LetT{f),T nig) be Blackwell games. If for any p S H, g{p) < 
val^(r(/,p)), and for any w ^ [H], g{w) < f{w), then vul^ {T nig)) < val^(r(/)). 
Similarly for the value and the upper value, and for > instead of < . 

Truncated subgames may be nested. If we have a nested series of truncated 
subgames, then we may extend a strategy for the smallest subgame to a strategy 
for all subgames. This allows us to approximate complicated games with a series 
of simpler, truncated subgames, obtain a strategy that is (e-)optimal in all the 
subgames. The final lemma in this section allows us to prove results for that 
strategy in the original game. 
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Definition 2.23 Let, for n G IN, fn be a payoff function, and Hn a set of 
stopping positions in r(/„). If for all n € IN, rH„{fn) is a truncated subgame 
oi^H^+iifn+i), and equivalent to TH„^i{fn+i) [for player I, II], then the series 
of games {Th„ ifn))neiN is called a nested series of equivalent truncated subgames 
[equivalent for player I, II]. 



Lemma 2.24 Let {^Hi{gi))ie]N be a nested series of truncated games equivalent 
for player I [player II]. Then all the games have the same lower value [upper 
value]. Furthermore, we can find a strategy for player I [player II] that is e- 
optimal in all the games Tniigi)- 



Sketch of proof: Basically, we apply Lemma 2.2C a number of times and 
use induction. The proof is straightforward, except for a slight complication 
involving the domain on which the strategies are defined. This complication is 
solved using the observations that if we truncate a game, any stopping position 
remains a stopping position, and that strategies can be assumed to be undefined 
on stopping positions. 

Corollary 2.25 Let (THi{gi))ii£]N be a nested series of equivalent truncated 
subgames. If^Hoido) determined, then all the games are determined, and all 
the games have the same value. Furthermore, we can find strategies for player 
I and player II that are e-optimal in all the games F/f^ (gi)- 



Remark 2.26 If the component games involved all have optimal strategies, 
then we can extend optimal strategies with optimal strategies to optimal strate- 
gies, i.e. drop the e in the above lemmas and corollaries. 



3 Determinateness Results 
3.1 Generalized Open Games 

In this subsection we prove determinacy of a class of 'generalized open games', 
where payoff for a play is calculated as the supremum of values associated with 
the positions hit in the play. In addition we derive a result for these and open 
games comparable to the compactness of W. 

Theorem 3.1 Let u : P ^ IR be a bounded function, and let f : W ^ Si be 
the payoff function defined by f{w) — sup^g^ u{wy). Then T{f) is determined, 
and 

val(r(/)) - lim val(r„(/„)) (3.11) 

n — >QO 

where /„(w) = supj<„ u{wy). 

Sketch of proof: Showing that lim„_>oo val(r„(/„)) exists and is not greater 
than the lower value of r(/) is not difficult. To show that it is not less than 
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the upper value, we approximate r(/) with a collection of finite auxiliary games 
^nig-n) such that the payoff at the stopping positions is an estimate of the 
value of the game at that point. We then show that these auxiliary games 
form a nested series of equivalent finite truncated subgames. This allows us 
to find a strategy that is optimal in each of the truncated subgames. This 
strategy is also a strategy in the game r(/), and has a value in r(/) equal to 
lim„^oo val(r„(/„)). 

Proof: Without loss of generality we may assume that the function u has range 
[0, 1]. For any p G P, and any n G M, the game r„(/„,p) is finite (of length 
< n), and thus determined. It is easily seen that /o < /i < /2 <•■•</< 1- 
Consequently, for any p G P, 

val(ro(/o,p)) < val(ri(/i,p)) < val(r2(/2,p)) < . . . < val-^(r(/,p)) < 1 

(3.12) 

For all p G P, lim/j_,oo val(rfc(/fc,p)) exists, since all monotone non-decreasing 
bounded sequences converge. Furthermore, for all p £ P, 

lim val(r„(/„,p)) < val^(r(/,p)) (3.13) 

n — >oo 

Define for any n £ IN the payoff function g„ : Wn [0, 1] by 

g„{p) = lim val(rfc(/fe,p)) for p e Wn (3.14) 

k—^oo 

Then for all p e Wn, gn{p) > val(r„(/„,p)) = /„(p). 

Furthermore, the games Tn{gn) form a nested series of equivalent truncated 
subgames. For fix n e IN, p G Wn- Define for k £ IN, : Wn+i IR by 

hn+i,k{p') = val(rfc(/fc,p')) for p' € Wn+i. Then 

gnip) = lim val{Tk{fk,p)) (3.15) 

k~^oo 

= lim val(r„+i(/i„+i.fc,p)) (3.16) 
= val(r„+i( lim /i„+i,fe,p)) (3.17) 

k—>-oo 

= val(r„+i(.g„+i,p)) (3.18) 



(equati on (3.16| ) follows from Corollary 2.21 , and equation ( ^.17 ) follows from 



Lemma 2.18 as Wn+i is finite). 



Since (r„(g„))„giv is a nested series of equivalent truncated subgames, by 



Corollary 2.25 the games r„(g„) all have the same value, say v. Also, we can 
find a strategy for player II that is e-optimal in all the games r„(g„), and since 
all the games Tn{gn) are finite and hence have optimal strategies, by Remark 



2.26 we can even find a strategy that is optimal in all the games r„(5„). So let 



r be such a strategy. Then for any strategy a, and any n G IV, 

E{a vs T in r„(.9„)) < val(r„(.g„)) = v (3.19) 
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Now let a be any strategy for player I in r(/). Then 

E{a vs T in r(/)) 

= lim E{a vs t in r„(/„)) (3.20) 

n^oo 

< lim E{(7 vs r in r„(5„)) (3.21) 

< V (3.22) 

So 

valT(r(/)) < val(r for player II in r(/)) < v (3.23) 

But also 

z; = val(ro(5o)) = ffo(e) = lim vaKYkifk)) < valJ'(r(/)) (3.24) 

k—*oo 

Therefore, 

valT(r(/)) = val'L(r(/)) = lim val(rfc(A)) (3.25) 

□ 

Corollary 3.2 Let O be an open set. Then T{0) is determined. 

Proof: There exists a set of positions H such that O = [H]. Then for all 



w € W, Io{w) — sup„g^ /if (win). Applying Theorem 3.1 yields the corollary. 

□ 

Corollary 3.3 Let O = IJ,- Oi be the union of open sets. Then val(r(0)) = 
lim„_ooval(r(U,<„O0)- 

Proof: As the union of open sets, O is open, and hence there is a set of positions 
H such that O = [H], i.e. 

O ^ {w eW \ 3p ^ H : p <Z w} (3.26) 

Define the basic open sets Bj C O by 

B,, = {w W \ 3p <^ H : p C w ^ len(p) < n} (3.27) 

then for all w 6 W , 

Io{w) = sup Ih{w\,) (3.28) 

IbAw) = suplniwy) (3.29) 

j<n 
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so applying Theorem 3.1, we find that 

val(r(0)) = hm val(r„(B„)) (3.30) 

n — >oo 

For each m £ £V, Bm is a closed set covered by the open sets {Oi)i^]N- So by the 
compactness of W there is for each m £ IN a Um £ such that Bm C Ur=ri Oi. 
Then for all n > n^, 

n 

val(r™(B„)) < val(r(|J O,)) < val(r(0)) (3.31) 

i=l 

The corollary follows immediately. 

□ 



Corollary 3.4 Let f be a continuous function. Then T{f) is determined. 

Proof: As W is compact, and / is continuous, f[W] is compact, and hence 
bounded. Define w : P — > by u{p) := inf^gjpj f{w). Then u is well-defined 
and bounded, and by t he continuity of /, f{w) = sup„g^ w(w|„) for all w G W. 
Applying Theorem 3T yields the corollary. 

□ 



Remark 3.5 In the case of open games (or generalized open games, described 
later) there is an optimal strategy for player II. This strategy can be described 
as 'at every position player II plays the optimal one-round strategy, looking at 
the values the game has for player I from all positions directly following that 
one'. However, for player I there does not always exist an optimal strategy, as 
the following example shows. 



Example 3.6 Consider the following Blackwell game. Each round, both play- 
ers say either 'Stop' or 'Continue'. If both players say 'Continue', then play 
continues. Otherwise, the game halts: player II wins (payoff 0) if both players 
said 'Stop', while player I wins (payoff 1) if only one of the players said 'Stop'. 
If play continues indefinitely, and neither player ever says 'Stop', then payoff is 
0, i.e. player II wins. 

This is clearly an open game. An interpretation of this game is, that player 
II tries to guess on which round player I will say 'Stop', and tries to match her. 
If player II guesses wrong, i.e. says 'Stop' too soon or not soon enough, then 
player I wins, if player II guesses right, then he wins. 

A strategy of value 1 — ^ for player I is, to select at random a number i 
between 1 and n, and say 'Stop' on round i. Translated to the standard format 
for strategies, this becomes: 

on round 1, say 'Stop' — of the time, 

on round 2, if not yet stopped, say 'Stop' ^^^^ of the time. 
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on round 3, if not yet stopped, say 'Stop' of the time, 

on round n, if not yet stopped, say 'Stop' j of the time. 

Hence, the value of this game is 1. In fact, the value of this game at any 
position in which game has not yet ended is 1. But there exists no optimal 
strategy of value 1. For suppose there exists such a strategy, of value 1. Then 
on any round (in which play has not yet ended), the chance that player I will 
say 'Stop' in that round is 0%. For otherwise, the strategy would not score 
100% against the counterstrategy that player II says 'Stop' on that round. But 
then, player I will never say 'Stop', and this strategy will lose against the coun- 
terstrategy that player II never says 'Stop'. So any strategy for player I has 
value strictly less than 1, although there are strategies with values arbitrarily 
close to 1 . This game is an example of a game in which one of the players has 
no optimal strategy. 



3.2 G^-sets 

Davis' proof of dcterminacy for G&a games of perfect information |^ is based 
upon the idea of 'imposing restrictions' on the range of moves player II can 
make. I.e. certain moves are declared 'forbidden', or a loss for player II, in such 
a way that (a) if player I did not have a win before, she does not get a win now, 
and (b) a particular Gs set is now certain to be avoided. By applying this to 
all the Gs subsets of a Gsa set, and using compactness, he shows that if player 
I cannot force the resulting sequence to be in one of the Gg sets, player II can 
force the resulting sequence to be outside all of them. 

The union of all the sequences in which one of the 'forbidden' moves is played, 
is an open set that contains the Gs set in question. One way of looking at Davis' 
proof is, that he enlarges each of the Gs sets to an open set without increasing 
the (lower) value of the game, in order to be able to apply determinacy of open 
games. 

In this subsection, we show that this holds (in a fashion) for Blackwell games, 
i.e. that a Gs set can be 'enlarged' to an open set without increasing the lower 
value of the game by more than an arbitrarily small amount, even in the presence 
of a 'background function', a payoff function for those sequences that are not in 
the Gs set. 

Theorem 3.7 Let / : ly — > [0, 1] be a measurable function and let D be a Gs 
set. Then 

val-L(r(max(/,/z5)))= inf val-L(r(max(/, /o))) (3.32) 

ODD,0 open 

Sketch of proof: We define a collection of auxiliary games F/f. (gi) of the 
game F(niax(/, /c)), in which the amount player I gets at a stopping po- 
sition p is an estimate for the value of F(max(/, /c)) at position p, namely 
mfoDD,o oponval(r(max(/,/o),p))- We then show that these auxiliary games 
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form a nested series of finite truncated subgames, equivalent for player I. This 
allows us to find a strategy that is e-optimal in each of the truncated subgames. 
This strategy is also a strategy in the game r(max(/, Id)), and has the required 
value, proving one side of the equation. The other side is trivial. 

Proof: Put V — inioDD.o open val'^(r(max(/, /q))). For any Gs set D we can 
find a set of positions H, such that D ^ {w E W \ <E H \ p C w} ^ oo}. 
We may assume that e G iJ. 



Define for any i e W, 

Hi := {p E H \ there are exactly i positions p' in H strictly preceding p} 

(3.33) 

Define for any i E IN the payoff functions 5^, hi by 

g,{p) = inf valJ'(r(max(/,/o),p)) forpeiJ, (3.34) 

O^D.O open 

g,{w) = f{w)itw^m (3.35) 

hiip) = IfovpeH, (3.36) 

h^iw) = fiw)iiw^[H,] (3.37) 

First, the games Tniigi) form a nested series of truncated subgames equivalent 
for player 1. For let i S IN, and fix p G Hi. Let O ^ D, then for any p' G 
such that p' 3 p, val^{T{max{f, Io),p')) > gi+i{p'), and for any w D p such 



that w ^ [Hi^i], max(/, /o)(w) > /(w) = gi+i{w). Hence by Corollary 2.22 
for any ODD, val-"^ (L (max (/, Jo),p)) > val^{TH,+A9i+i,P))- Therefore, 

gdp)>ys^lH^H,^^{g^+up)) (3.38) 

On the other hand, for any e > we can find, for each p' G Hi+i, an open set 
Op' 3 D such that 

val^(r(max(/,/o^,),p')) < g,+i{p') + e (3.39) 

Set O = Up'eH,+,([p'] nOpO- Then for all e valJ'(r(max(/, /o),p')) = 

val''^(r(max(/, /Op,),p')) < gi+i(p')+ e and for any w ^ [H+i], max(/, /o)(u;) = 
f{w) = gi+i{w). Hence by Corollary 2.22| , 

g,ip) < valJ'(r(max(/,/o),p)) < valHrH,^A9^+l,P)) + e (3-40) 

This holds for any e > 0, therefore 

g,(p) =val'l(rH.+i(g.+i,p)) (3.41) 

Finally, for any i G IN, and any play w ^ [Hi], we have that w ^ [if^+i], 
and hence gi{w) = /(w) — gi+i{'w). So F//. (g^) is a truncated subgame of 
^Hi+iigi+i) equivalent for player 1. 
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Let e > 0. Since (F/f. ((7i))igjv is a nested series of truncated subgames equiv- 
alent for player I, by Lemma p.24| all the games have the same lower value, 
namely val^{THoigo)) = 5o(e) = and there exists a strategy a for player I 
that is e-optimal in all the games T^i i-^- fo'^ ^'^y strategy r, and any i G iV, 

E{a vs r in FH.Cg,)) > val^(rH.(gO) -e^v-e (3.42) 

Now let T be any strategy for player II in r(max(/, //))). Then 

E{a vs r in r(max(/, Id))) 

= lim £'(cr vs T in r//^(/i,)) (3.43) 

> lim E{a vs r in Tn^igi)) (3.44) 

> v-e (3.45) 

So CT is a strategy for player I of value at least v — e. This implies that 
val^(r(max(/, /£)))) > v — e. This construction can be done for any e > 0, 
hence 

valHr(max(/,/i3))) > v (3.46) 
For any ODD, val^(F(max(/, /d))) < val^(F(max(/, /o))), hence 

val^(F(max(/, /£,))) < inf val^ (F(max(/, /q))) = v (3.47) 

OQD,0 open 

Hence val'''(F(max(/, /£>))) = v. 

□ 



Corollary 3.8 Let S be a measurable set, and let D be a Gs set. Suppose that 
T{S U D) has lower value v. Then for any e > 0, there exist an open set O, 
D C_ O, such that T{S U O) has lower value at most v + e. 



Proof: Take f = Is and apply the non-trivial part of Theorem B.7 



□ 



Corollary 3.9 Let D be a Gs set. Then T(D) is determined, and 

val(F(i?)) = inf val(F(0)) (3.48) 

ODD.O open 

Proof: For any open set O ^ D, F(0) is determined and val^(F(Z?)) < 
valT(F(i:>)) < val(F(0)). Applying Theorem 1^ with / = yields the Corollary. 

□ 
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3.3 G^^-sets 



In this subsection, we prove the determinacy of r(/) in the case that / is 
the indicator function of a Gsa set. Structurally, this proof is similar to the 
aforementioned proof by Davis for Gsc^ games of perfect information We 
apply the results of the previous subsection to the Gs subsets of a Gsa set. 
Corollary 3.3 takes the place of the compactness used in Davis' proof. 

Theorem 3.10 Let S — IJi -^i Gsa set. Then r{S) is determined. 

Sketch of proof: Each of the Gs sets composing the Gsa set is enlarged to 
an open set using Corollary |3.8| , in such a way that at all times the lower value 
is not increased by more than e (compared to the original game), where e is 
arbitrarily small. The resulting union of open sets is itself open, and hence 
determined, and furthermore Corollary 3.3 allows us to conclude that the total 
increase of the lower value is still not more than e. This means that the upper 
value of the original game is also only at most e more than the lower value. 

Note that, unlike the previous proofs, this proof does not produce an optimal 
or e-optimal strategy. 



Proof: Put w = va iHr(S')). Let e > 0. Using Corollary O, we can find 
inductively open sets Oi D Di such that for all j £ IN, 



valHr(S'U y 0,;))<valJ'(r(5u|j0,)) + e/2^' 

i<j + l i<j 



Then for all j G W, 



val^(r(S'U IJO,)) <v + e 

i<j 



and hence, for all j Si IN, 



val(r(|J O,)) <v + e 



Then by Corollary 3.2, 



val(r( U O,)) <v + e 

Since S = [j^^j^ D^ 0^, 

valf (r(5)) < valf (r( y O,)) = val(r( U O,)) <v + e 

ielN ielN 

This is true for any e, hence val^(r(5)) = w = val^(r(5)). 



(3.49) 



(3.50) 



(3.51) 



(3.52) 



(3.53) 



□ 
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Remark 3.11 The proof of Theorem 3.1C shows that any Gsa set (and a fortiori 
any set of lesser complexity) can be enlarged to an open set such that the value of 
the Blackwell game on that set is not increased by more than an arbitrarily small 
amount. A plausible conjecture is, that this holds for any Borel-measurable set. 

This conjecture holds in the case of games of Perfect Information. Such a 
game, on a Borel-set S, is determined and has value or 1. If it has value then 
player II has a winning strategy. The set of plays that cannot occur if player II 
uses that strategy, is an open set, and the game on that set has value as well. 



4 The Axiom of Deter minacy for Blackwell Games 

For Games of Perfect Information, there exists the Axiom of Determinacy, which 
states that any Game of Perfect Information with finite choice of moves is de- 
termined^. AD has many interesting consequences, such as the existence of an 
ultrafilter on Hi, the existence of a complete measure on IR, the non-existence 
of a sequence of Hi reals, and the negation of the Axiom of Choice. We can 
formulate an analogue of AD with respect to Blackwell Games, and look at the 
consequences of that axiom. But AD is an axiom about games on all subsets of 
W, not just on the Borel measurable subsets]] An analogous axiom for Black- 
well Games should therefore not be limited to games with measurable payoff 
functions. Hence we need to extend the concepts of expectation and value for 
Blackwell games. 

Definition 4.1 Let r(/) be a Blackwell Game, where / is bounded but not 
necessarily Borel measurable. Let a and r be strategies for players I, II. a and 
T determine a probability measure /io-.r on W , induced by setting 

n 

Ma,rN = P{w I w hits p} = J]^ (cr(p|(i_i))(a;^) • T{p\(i_i)){yi)) (4.1) 

1=1 

for any position p ^ {xi,yi, . . . ,Xn, y„) G P. 

Instead of the expected income of player I, if she plays according to cr and 
player II plays according to r, we now have the lower and upper expected income 

E~{a vs T inT{f)) = sup g{w)d^ia,Tiw) (4.2) 

g^f^g measurable J 

£;+(cr VS r in r(/)) = inf [ g{w)dfia.r{w) (4.3) 

g^f^g measurable J 

^ Formally, AD is an axiom about games with countable choice of moves, whose payoff 
function is the indicator function of a set S QW. But in the case of Games of Perfect Infor- 
mation, determinacy for games with countable choice of moves is equivalent to determinacy for 
games with finite choice of moves, and determinacy for games with 0-1-valued payoff functions 
is equivalent to determinacy for games with arbitrary bounded payoff functions. 

^ AD with respect only to gjames on Borel measurable subsets (and finite sets X and Y) is 
in fact provable from CAC |^,^]- 
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Lower value and upper value are redefined in the obvious way: 

val^(r(/)) = supinf £;"(cr vs T in r(/)) (4.4) 
val'f(r(/)) = infsup£;+(cr vs r in r(/)) (4.5) 

Note that in the case that / is measurable, these definitions reduce to the old 
definitions. 



Definition 4.2 The Axiom of Determinacy for Blackwell Games (AD-Bl) is the 
statement that for every pair of non-empty finite sets X, Y, and every bounded 
function f on W — {X x Y)-"^ , the Blackwell Game r(/) is determined, i.e. 

valHr(/)) -valT(r(/)) (4.6) 



Theorem 4.3 Assuming AD-Bl, it follows that all sets of reals are Lehesgue 
measurable. 

Sketch of proof: Let X and Y be the set {0, 1}. Then we can construct a 
mapping (j) : W ^ [0, 1] such that if either or r is the strategy that assigns the 
^-^ probability distribution to every position, then the measure ^a-,T induces 
the Lebesgue measure on [0, 1]. We can then deduce the measurability of a set 
"5* ^ [0, 1] from the determinacy of the game T{(f)^^[S]). 



Some of the consequences of Theorem 4.3 are, that AD-Bl is not consistent 



with AC, and that the consistency of ZF -I- AD-Bl cannot be proven in ZFC. 
These results are all similar to results for AD. An open problem is that of 
the relationship between AD and AD-Bl, whether AD follows from AD-Bl, or 
vice versa, or even whether AD-Bl follows from a stronger version of AD such 
as ADjR. From a given game of Perfect Information, we can easily construct 
a Blackwell game that is 'equivalent', and assuming AD-Bl we can find an e- 
optimal mixed strategy for that equivalent Blackwell-game. However, to derive 
AD from AD-Bl, we need to have a pure strategy, and even though we can inter- 
pret any mixed strategy as a probability distribution on pure strategies, there 
is no guarantee that any of these pure strategies will do as well as the mixed 
strategy against all counterstrategies. 
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A Notational Conventions 

• I, II: the two players 

• X,Y: the two sets out of which I and II choose their moves 

• Z: X xY 

• w^w': plays 

• W: the set of all plays 

• Wn- the set of all finite plays of length n 

• S: a, subset of W 

• B: & basic open subset of W 

• O: an open subset of W 

• D: a Gs subset of W 

• p,p': finite plays or positions 

• e: the starting position 

• P: the set of all positions 

• H,H': sets of positions, usually stopsets or stopsets-to-be 

• w D p: the play w hits position p 

• p' Z) p: the position p' follows p 

• p' ^ P'- the position p' follows or is equal to p 

• [p] : the set of plays hitting position p 

• [H]: the set of plays hitting any position in H 

• p^p' ,p^w: the plays obtained by concatenating sequences p and p' or w 

• P\m the plays consisting of the first n moves in p or u; 

• len(p): the length of a position p 

• /, g, h: payoff functions 

• f{w): the payoff player I gets from player II for the play w 

• f{p): if p is a stopping position, the payoff player I gets from player II for 
any play w that passes through p 

• r(/): the game with payoff function / 
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• Tnif)- the game with payoff function / and stopset H 

• r„(/): the finite game with payoff function / and stopset Wn 

• r(5): the game with payoff function Is 

• r(/, p): the game with payoff function /, starting from position p 

• a: a strategy for player I 

• r: a strategy for player II 

• E{a vs r in r(/)): the expected payoff when playing a against r in the 
game r(/) 

• val(CT in r(/)): the value of cr in the game r(/) 

• val(r(/)): the value of the game r(/) 

• val^(r(/)): the lower value of the game r(/) 

• val^(r(/)): the upper value of the game r(/) 

• u,v: values 

B Proofs 

B.l Equivalent Truncated Subgames 



Lemma |2.20| Let T{f) be a Blackwell game, and let Tnig) be a truncated sub- 
game of r(/), truncated at a set of positions H , equivalent for player I [for 
player II]. T/ien val^(r(/)) = val^(rff (5)) [val^T{f)) ^ val^THig))]- Further- 
more, for any e > 0, any e-optimal strategy for player I [player II] in Tnig) (if 
it is undefined on all positions at or after positions in H) can be extended to an 
e-optimal strategy for player I [player II] in T{f). 

Proof: Let r(/) be a Blackwell game, and let r^f(g) be a truncated subgame 
of r(/), truncated at a set of positions H, equivalent for player I. This means 
that for any p £ H, g{p) = val'''(r(/,p)), and for any w ^ [H], g{w) = /(w). 

Let e > 0, and let (Tq be an e-optimal strategy for player I in the game Vnig)- 
If u = val^(r^f (g)), and u — val(cro in ^H[g)), then Q < v — u < e. So choose 
(5 > such that 5 < e ~ {v — u), and fix for each p e iJ a (5-optimal strategy 
<7p for player I in T{f,p). By Remark 2.14| we may assume that for any p £ H, 



ap is defined exactly on p and those positions that are after p, and that cto is 
defined exactly on those positions that are not at or after any position in H. It 
follows that a = Upeif U ctq is a well-defined strategy for player I in the game 

r(/). 
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Now let r be a strategy for player II in r(/). Then for each p G H, 

/ f{w)df,,,rH = p^^ii—-— U.,x[p] (B.l) 

= / /(w)c?/VTinr(/.p)(w) /(Xa,rb] (B-2) 
\JwG[p] J 

= £;(avsTinr(/,p)K,,[p] (B.3) 

= E{ap vs T in T{f,p))fi^„^r[p] (B.4) 

(val^(r(/,p))-<5)/z,„,,[p] (B.5) 

(!?b)-<5)Mao,Tb] (B.6) 



> 



and consequently, 

E{a vs T in r(/)) 

= / f{w)dn„,r{w) (B.7) 

= I f{w)d^i,^r{w) \ + / (B.8) 

> y^iaip) - ^)I^<7o,t\p]+ 9{w)diicr^^r{w) (B.9) 



> y"ff(PKo,r[p]+ / ^H^Mao.rH (B.IO) 



= / 9{w)diJao,T{w) - 6 (B.ll) 

> u-6 (B.12) 

> v-e (B.13) 

So (7 is an extension of cto of value greater than v — e. Since this can be done 
for any e > 0, and any e-optimal strategy ao, this implies that val^(r(/)) > v. 

Similarly, let e > 0, and let ct be a strategy for player I in r(/). Then wc can 
find counterstrategies tq, Tp forp e H for player II in Tnig), r(/,p) respectively, 
such that E{a vs tq in Tnig)) < val^(r//(g)) + e/2, and E{a vs Tp in T{f,p)) < 
val^(r(/,p)) + e/2 for any p G H. We can assume that for any p G H, Tp is 
defined exactly on those positions that are at or after p, and that tq is defined 
exactly on those positions that are not at or after any any position in H. It 
follows that T = UpGff TpUTo is a well-defined strategy for player II in the game 
r(/). Then for each pGH, 



I 



f{w)diia,T{w) = I — IMt,t[p] (B.14) 
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f{w)d^i„,^ in r(/,p)(w) /i<T,rb] (B.15) 

= S(fTVSTinr(/,p)K,r[p] (B.16) 

= £:(cr vs Tp in r(/,p))^^,^Jp] (B.17) 

< (val^(r(/,p)) + e/2) b] (B.18) 

= (g(p) + e/2)Ma,ro[p] (B.19) 

and consequently, 

£;(cr vs r in r(/)) 

/(w;)dMa.rH (B.20) 



= Y.\ I f{w)dii,A'>J^)\+ I /HdA^.-rH (B.21) 
< I](ff(p) + e/2W.rob]+ / giw)dt,„,roiw) (B.22) 



< y2g{p)na.ro[p]+ / g{w)dfia,ro{w)+e/2 (B.23) 



g{p)dfi^^M + e/2 (B.24) 

< u + e/2 + e/2 (B.25) 
= v + e (B.26) 

Since this can be done for any strategy a and any e > 0, it follows that 
val^(r(/)) < V. Hence 

val^(r(/))=t; = valHrH(5)) (B.27) 

An e-optimal strategy for player I in Tnig) can be extended to an e-optimal 
strategy for player I in r(/) as in the first part of the proof. 
The proof for the upper value is analogous. 

□ 



Theorem 2.24 Let (THi{gi))ieiN be a nested series of truncated games equiv- 



alent for player I [player II]. Then all the games have the same lower value 
[upper value]. Furthermore, we can find a strategy for player I [player II] that 
is e-optimal in all the games Tniigi)- 

Proof: For alH G iV, THi{gi) is a truncated subgame of F/^.^j (5^+1), equivalent 
for player I. By Lemma 2.20| , this means that for all i e _£V, val^(F^f; (g.;)) = 
\al^{TH,+i{gi+i))- It follows that for all i e W, 



valHFH,(.9.))=val^(FH„(5o)) 



(B.28) 
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Now let e > 0, and let cto be an e-optimal strategy for player I in Tho{9o)- 
Assume that ctq is only defined on non-stopping positions of ^Hoigo)- We can 
inductively define e-optimal strategies (Tj in the games Tj^.^gi) such that for all 
i G IN, (Tj+i is an extension of (Tj. For suppose we have extended CTq to an 
e-optimal strategy ai in TH.{gi), defined only on non-stopping positions. Then 



Ui is undefined on positions at or after positions in Hi, and thus by Lemma 2.20 
we can extend dj to an e-optimal strategy ct,-,),]^ in THi+i (di+i)- 

By Remark we may restrict this strategy <jI_^i to the non-stopping 

positions in THi^i{gi+i)- This restriction di+i is an extension of cr^ as well. 
For let p be a stopping position in THi^iigi+i)- If P is preceded by a stopping 
position in Tniigi), then p is itself a stopping position in Tniigi)- Otherwise, 
we observe that for any play w D p, gi+i{w) = gi+i{p), and for any position 
<1 ^ P^ ^{gi+i^l) is determined with value gi+i{p). By the truncated-subgame 
condition, this implies that any w G [p] has payoff gi+i{p) in THi{gi), and again 
p is a stopping position. 

Now, if p is a position, and cri(p), crj(p) are defined, i < j, then ai{p) — 
fTi+i(p) = ... — aj{p). Therefore we can define a hy a — Uiew '^i' 

a{p) — <Ji{p) if a.i is defined on p, for any i G IN (B.29) 

Then for any i 6 IN, for any non-stopping position p of ^Hiigi), is defined 
and (t{p) — (Ti{p). Hence, a is e-optimal in all the games, a is defined on all the 
non-stopping positions of all the games F//. (17^), but if necessary we can extend 
(T to a strategy a' defined on all positions, by choosing a default probability 
distribution for those positions on which a is not defined. 

□ 



B.2 Finite Games 

Lemma B.l Let C be a closed convex set in M", and let b G -S?" — C. Then 
there exists a hyperplane separating b and C , i.e. a vector y G iR" — {0} and 
d G IR such that 

y'^b > d (B.30) 
y'^z < d for each z £ C (B.31) 

Proof: Since the theorem is trivial if C = 0, we assume C As C is closed, 
there exists a vector c in C that is nearest to b, i.e. that minimizes ||z — And 
because b^C, \\c - b\\ > 0. We define 

y = b-c (B.32) 
d = l(y^fe + z/c) (B.33) 



Then - d = y'^{b - i(6 + c)) = - c)^{b - c) > 0, proving that O.30D 
holds. 
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Now suppose that (B.31) docs not hold. Then for some z € C, z > d. 
Since, y^c — d = y^{c — ^{b + cj) — —^{b — c)'^{b — c) < 0, this iniphes 
y'^{z — c) > 0. Hence there exists a A with < A < 1 and 



A < 



2y'^{z - c) 



||z-cP 

As C is convex, c+ A(z — c) belongs to C. Moreover, 



||(c + A(z-c))-6|| 



||A(z 



yf 



(B.34) 



(B.35) 



X^\\z-c\\'-2Xy^{z-c) + \\y\\' (B.36) 



< 



(B.37) 



contradicting the fact that c is a point in C nearest to b. Therefore, for all 
z E C, y^ z < d, and (B.31) holds. 



□ 



Theorem |2.16| (Von Neumann's Minimax TheoremJT2[) Let Ti{f) be 



finite one-round Blackwell game (i.e. of length 1). Then Ti[f ) is determined, 
and both players have optimal strategies. 

Proof: / is (or can be interpreted as) a function X x Y ^ IR. Without 
loss of generality we may assume that X = {l,...,n},Y — {l,...,m}. A 
strategy a for player I in ri(/) is (or can be interpreted as) a nonnegative 
vector (xi, . . . , Xn) such that J2^=i — 1^ ^^'^ similarly, a strategy r for player 
II is (or can be interpreted as) a nonnegative vector (yi,...,?/™) such that 
X]j=i yj = 1- It is easily seen that 

n rn 

E{avsTmr,{f)) - ^ ^ x,2/,/(i, j) (B.38) 

i=l 3 = 1 

n 

val(a inri(/)) = min VxJ(i,j) (B.39) 

l<j<m ^ — ^ 
i—1 

m 

val(r inri(/)) = max ^ y,/(i, j) (B.40) 

l<i<n ^ — ^ 
~ " J = l 

Let C C M™ be the convex hull of the vectors (/(i, 1), . . . , f{i, m)), i ~ 1, . . . ,n. 
Then each point z E C corresponds to (at least) one strategy <j for player I, with 
val(cr in ri(/)) being equal to the coordinate of z which has the least value. Let 
— {z E -K™ I 3z' G C : z < z'}, using coordinatewise ordening of vectors. 
Then player I has a strategy of value u E IR iS the vector {u, . . . ,u) G . It is 
obvious that C and C" are both closed convex sets. 

Now let V — val'''(ri(/)), e > 0. Then there exists no strategy a for player 



I of value V + e, i.e. b — {v + e, . . . , v + e) ^ C . By Lemma B.l this implies 
that there exist a vector y e _ZR™ — {0} and d E M, such that y^ b > d while 
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z < d for any z G C . For any j < m, any z G C , and any r > 0, 
y'^z-yjr = . . . ,Zj-i, Zj-r, Zj+i, . . . ,z„) < d, i.e. > {y^z-d)/r. Since 

r > is arbitrary, it follows that for any j < to, yj > 0. Since y ^ 0, we may 
also assume, without loss of generality, that X^jli Vj = 1- Then for any z G C^, 
y-^z < d < y'^b — v + e. In particular, for i = 1, . . . ,n, < + £■ 

Thus, ?; corresponds to a strategy r of value lower than v + e. Since we can do 
this construction for any e > 0, it follows that val^(ri(/)) — v = val^(ri(/)). 

The existence of an optimal strategy for player I follows from the observation 
that C is a closed and bounded subset of iR™ (and hence compact), and that 
the function min : JR™ M taking the minimum coordinate is continuous. A 
similar argument yields the existence of an optimal strategy for player II. 

□ 



Theorem 2.17 Let r„(/) be a finite Blackwell game of length n. Then r„(/) 



is determined, and both players have optimal strategies. 

Proof: We will proof this by induction on the length n of the games. 

For n = 0, determinacy is trivial. So let n > 0, let r„(/) be a finite Blackwell 
game of length n, and suppose that all finite Blackwell games of length m < n 
have already been shown to be determined, and to have optimal strategies. For 
each position p G Wi, the game T{f,p) is finite of length < n — 1. Hence, by 
the induction hypothesis, each of those games is determined and has a value 
val(r(/, p)). Define the payoff function g -.W ^ Mhy 



ff(p)=val(r(/,p)) for pG 1^1 



(B.41) 



Then by Von Neumann's Minimax Theorem, the game ^i{g) is determined. 
^i{g) is by its definition an equivalent truncated subgame of r„( /), so by 
Corollary 2.21 , r„(/) is determined. Furthermore, by Remark 2.26| the opti- 
mal strategy produced by Von Neumann's Mini-Max Theorem can be extended 
to an optimal strategy in r„(/). 

□ 



B.3 Lebesgue Measurability from AD-Bl 

Theorem |4.3| Assuming AD-Bl, it follows that all sets of reals are Lebesgue 
measurable. 

Proof: It suffices to show that the Lebesgue measure on [0, 1] is complete. Set 
X = Y = {0, 1}, and define (/) : ^ [0, 1] by 

OO 

</.((a:i,yi,X2,y2,...)) = 5I2"'(a;^©l/^) (B.42) 
1=1 

where 0©0=1©1 = 0, 001 = 100:= 1. Now let cr,T be strategies, 
and suppose that one of those strategies is the strategy that assigns the ^-^ 
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probability distribution on X or Y, respectively. Then for any i G IN, Xi © j/i 
has equal chances of being or 1, and the distribution of (/'(w) on [0, 1] is the 
uniform distribution on [0,1] under the Lebesgue measure on [0,1]. It follows 
that for any subset S C [0, 1], 

^'"""cr,r(0-i[S']) = Ma ""'(5') (B.43) 
Ai°"*"a,T(0-i[5]) = iJiT"{S) (B.44) 

where SUp^cA measurable (-8), /^°"*'''(^) = inffiDA measurable 

Let S C [0, 1]. No strategy for player I in the game r((/)~^ [S']) can have value 
greater than m™"®''(S'), since this is the lower expected income of any strategy 
for player I against the i-i strategy. Similarly, no strategy for player II in the 
game r(0^^[S']) can have value less than /i™*°''(S'). Therefore 

val^(r(0-i[^])) < /ir"(^) < P-T^^S) < valT(r(0-i[5])). (B.45) 

From the determinacy of the game r(0~^[S']), it now follows that 

Air-(5) = Mr°'^(5) (B.46) 

Since this holds for arbitrary sets S C [0, 1], all subsets of [0, 1] are Lebesgue 
measurable. 

□ 



C The Locally Optimal Strategy 



As can be seen by inspection of the proof of Theorem ^.17 , an optimal strategy 



in a finite game is to use, at each position p, Von Neumann's Minimax Theorem 
to generate the probability distribution for that position p. We do this by 
calculating, for each position p, the optimal strategy for the one-round game 
whose payoffs are the values of the games starting at the positions directly 
following p. 

In a non-finite game, we can do the same thing, i.e. for a given position p, 
we can apply Von Neumann's Minimax Theorem to the one-round game whose 
payoffs are the lowervalues of the games starting from the positions directly 



following p. By Lemma 2.20, the value of this one-round game is equal to the 
lowervalue of the game starting at p, giving us a relationship between lower- 
values at different positions. In an open game, for player II, the strategy of 
always using this 'locally optimal' probability distribution is an optimal strat- 
egy (Remark |3.5| ). But this does not hold in general: see also Example |3.6| for 
a counterexample. 

As to why this 'local optimality' does not translate to optimality for the 
whole strategy: a single probability distribution that is optimal in the corre- 
sponding one-round game would be optimal in the whole game, if for any posi- 
tion p following the current position, the strategy used in the subgame T{f,p) 
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were optimal. But in order to use this to show that each of the separate prob- 
ability distributions in the 'locally optimal strategy' is optimal in the whole 
game, we would first need that the strategy as a whole is optimal in entire sub- 
games, and for that we need that each of the separate probability distributions 
is optimal in those subgames, and for that we need that the strategy as a whole 
is optimal in entire subgames of subgames, etcetera, etcetera, ad infinitum. In 
the case of finite games this eventually reduces to subgames of length 0, which 
are trivial, but in the case of infinite games the proof falls through. 

In general, it is impossile to construct an optimal or e-optimal strategy 
merely from the values a game has at each of its positions, as the following 
example shows. 

Example C.l Consider the following two Blackwell games. Player II generates 
a sequence of O's and I's. In the game r(S'i), player II wins if he generates 
infinitely many I's. In the game r(52), player II wins if he generates only 
finitely many I's. Player I has no influence over the outcome of the game. 
Payoff is 1 if player I wins, if player II wins. 

It is clear that player II can win from any position, in both games. Hence 
both games have lower and upper value 0, starting from all positions. But any 
strategy for player II that is good in r(S'i), will be bad in r(S'2), and vice versa, 
since the two sets of winning positions are complementary. Hence there cannot 
be any method of finding optimal or good strategies that merely uses the values 
of a game. 

Note that here we have two sets of winning positions Si and S2, such that 
r(S'i), r(52) have value (in all positions), and r(S'i U ^2) has value 1 (in all 
positions). 
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